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Abstract 

In this work, we consider two types of adversarial attacks on a network of nodes seeking to reach 
consensus. The first type involves an adversary that is capable of breaking a specific number of links at 
each time instant. In the second attack, the adversary is capable of corrupting the values of the nodes by 
adding a noise signal. In this latter case, we assume that the adversary is constrained by a power budget. 
We consider the optimization problem of the adversary and fully characterize its optimum strategy for 
each scenario. 



I. Introduction 

Starting with the work of JTJ, distributed computation received increased attention. The core 
idea behind various distributed decision applications is the ability of individual agents to reach 
agreement globally via local interactions. Fields where this idea is key include flocking and 
multiagent coordination [|2|-[|4), optimization (5J, [|6j, and the study of social influence [|7). 

As a particular example, consensus averaging involves agents who attempt to converge to 
the average of their initial measurements through local averaging. Consensus problems can be 
formulated in both discrete and continuous time. Consensus protocols find many applications 
in sensor networks where sensors collaborate distributively to make measurements of a certain 
quantity, such as the temperature in a field. The convergence of consensus algorithms has been 
studied widely; see |2). 

*This work was supported in part by an AFOSR MURI Grant FA9550-10-1-0573. 



March 13, 2013 



DRAFT 



2 



The convergence of consensus protocols under the effect of non-idealities have also been 
studied in the literature. [[8j study the convergence properties of pairwise gossip under the 
constraint that agents can only store integer values. Consensus in networks with noisy links 
was explored by [9] and [10|. p"TJ consider the case where the nodes are allowed to be mobile. 



The effects of switching topologies and time delays were considered in [ |r2| , [jT3J, and p4|-p6|. 

Here, we study the problem of continuous-time consensus averaging in the presence of an 
intelligent adversary. We consider two network- wide attacks launched by an adversary attempting 
to hinder the convergence of the nodes to consensus. The adversarial attacks we explore here 



differ from the ones studied by [ [17] , [ [T8| , and (T9J, who consider the effect of malicious 
and compromised agents who could update their values arbitrarily. In the first scenario (called 
Attack- I) we consider, the adversary can break a set of edges in the network at each time 
instant. In practice, the adversary would be limited in its resources; we translate this practical 
limitation to a hard constraint on the total number of links the adversary can compromise at each 
time instant. In the second case (called Attack-II), the adversary can corrupt the measurements 
of the nodes by injecting a signal under a maximum power constraint. Our goal is to study the 
optimal behavior of the adversary in each case, given the imposed constraints. 

For both attacks, we formulate the problem of the adversary as a finite horizon maximization 
problem in which the adversary seeks to maximize the Euclidean distance between the nodes' 
state and the consensus line. We use Pontryagin's maximum principle (MP) to completely 
characterize the optimal strategy of the adversary under both attacks; for each case we obtain a 
closed-form solution, providing also a potential-theoretic interpretation of the adversary's optimal 
strategy in ATTACK-I. Furthermore, we support our findings with numerical studies. 

In Sect|II} we describe ATTACK-I and formulate and solve the adversary's problem. We study 



Attack-II in Sect IIP present simulation results in Sect IV and conclude in SectLV 



II. Attack I: An Adversary Capable of Breaking Links 

Consider a connected network of n nodes described by an undirected graph Q = (J\f,£), 
where J\f is the set of vertices, and £ is the set of edges with \£\ = m. The nodes of the network 
are the vertices of Q, i.e., \M\ = n; we will denote an edge in £ between nodes i and j by (i, j). 
The value, or state, of the nodes at time instant t is given by x(t) = [xi(t), x n (t)] T . The nodes 
start with an initial value x(0) = xq, and they are interested in computing the average of their 
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initial measurements x avg = ^ Y^=i x i(®) y i a l° ca l averaging. We consider the continuous-time 
averaging dynamics given by 



where the rows of the matrix A(t) sum to zero and its off diagonal elements are nonnegative. 
We also assume that A(t) is symmetric. Further, we define x = lx avg and let M = A 
well known result states that, given the above assumptions, the nodes will reach consensus, i.e., 

lim^oo x(t) = x. 

An adversary attempts to slow down convergence. At each time instant, he can break at most 
I < m links and wishes to select the links which will cause the most harm. Let G {0, 1} 

be the weight the adversary assigns to link He breaks link at time t when = 1. 

His control is given by u(t) = [u 12 (t), u 13 (t), u ln (t), u 23 (t), u (n ^ 1)n (t)] T . If £ £,then 
Uij(t) = for all t. We will denote the number of links the adversary breaks at time t by N u (t). 
Then, N u (t) = \u(t)\ 2 , where |.| denotes the Euclidean norm. In accordance with the above, the 
strategy space of the adversary is 



Jo 

where the kernel k(t) is positive and integrable over [0,T]. The adversary's problem can be 
formulated as follows: 



x(t) = A(t)x(t), x(0) = x , 




The objective function of the adversary is: 



J(u) 




max u(t)eW 



J{u) 



(1) 



n 



S.t. 



x(t) = A(t)x(t), A u (t) 



Aij(t) = Aji(t) = a,ij (1 - Uijit)) , 



a i:j > 0,aij > ^ e 8. 



The Hamiltonian is then given by: 



H(x,p, u ) = k(t) \x(t) - x\ 2 + p(t) T A(t)x(t). 
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The first-order necessary conditions for optimality are: 

d 

P(t) = ~fa. H ( x >P> u ) 

= -2k(t)(x(t) -x)- A(t) T p(t), p(T) = (2) 

x(t) = A(t)x(t), x(0) = x (3) 

it* = aigmax{H(x,u,p, t) : u E U} , 

where x(t),p(t) E C 1 ^, T], the space of continuously differentiable functions over [0, T]. Let 
& x (t, 0) and 0) be the state transition matrices corresponding to x(t) and respectively. 
Then, the solutions to ODEs and Q are: 

a?(t) = ^(t,0)x , (4) 

p(t) = $ p (t, 0)p(0) - 2 / [%{t,r)k{r){x{r) - x)]dr. 



o 



Using the terminal condition p(T) = 0, we can write: 

p(0) = 2 / $ p (0, r)fc(r) [$ x (r, 0)x - x] dr. 
J o 



We therefore have 

-T 



P(t)= 2 j t $p(t,T)k(T)[$ x (T,0)x O -x]dT. (5) 



Let us write 



i=i \j=i 



n 



Y)Pi(t) I - a ij u ij{ t ) x i{ t ) + a ij U ij( t ) X j( t ) 

i=l \ j=l,j^i j=lj¥=i 

n n 

Uij{t)aijPi(t)(xj(t) - Xi(t)) 

i=l j = l,j^i 
n j-1 

j=2 i=l 



We further have 



max H(x,p,u) = max fc(t) |x(t) — x| + p(t) T A(t)a;(t) 
u(t)eu u(t)eu 

n j-i 

= k(t) \x(t) ~ x \ 2 + YY U 

j=2 i=l lJ 
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where fij(A,x,p) = a^ipjit) -pi(t))(xi(t) - Xj(t)). Let (f 1} f m ) = ir(f) be a nondecreasing 
ordering of the fi/s. Define the subset of edges % as follows: X t = {(i, j) G M : < and < fi+i}. 
Further, let I t be the set containing the i elements of % having the smallest values. Hence, we 
conclude that the optimal control is: 

l if {i,j)e It 

u%{t) = < if fij > (6) 

{0,1} if 4 = 

v 

The functions depend on both the state and the co-state, which in turn are defined in terms of 
the control. This makes it hard to obtain a closed-form solution for the control. However, in the 
following we consider the utility of the adversary which allows us to completely characterize his 
optimal strategy; we will be using the term "connected component" to refer to a set of connected 
nodes which have the same values. Let Wij(t) := aij(xj(t) — Xi(t)) 2 . 

Theorem 1: For all t, the optimal strategy of the adversary, u*(t), is to break £ links with 
the highest Wij(t) values. Furthermore, if the adversary has an optimal strategy of breaking less 
than £ links, then either Q has a cut of size less than £ or the nodes have reached consensus at 
time t. In either of the cases, breaking £ links is also optimal. 

Proof: We first characterize N u *(t), for all t. Because x(t),p(t) G C^T], the value of 
fij cannot change abruptly in a finite interval. As a result, the control obtained from the MP 
cannot switch infinitely many times in a finite interval. To this end, let [s, s + As], As > 0, be 
a small subinterval of [0, T] over which the adversary applies a stationary strategy u A such that 
N u a < £ with a corresponding system matrix A. Because the control strategy is time-invariant, 
the state trajectory is given by 

x(t) = e A{t - s) x{s), te[s,s + As}. 

Let P{t) := e At . Due to the structure of A, P(t) is a doubly stochastic matrix for t > 0; see 
(20), p. 63. 

Note that we can write x(s) = Px , where P is some doubly stochastic matrix. Indeed, 
assume that the control had switched once at time s G [0, s), and that the system matrix over 
[0, s) was At, and the system matrix corresponding to [s, s) was A 2 . Then x(s) = e A2( - s ~ s ^e Als xo. 
Because both e Alt ,e A2t are doubly stochastic matrices, their product is also doubly stochastic. 
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We can readily generalize this result to any number of switches in the interval [0, s). With this 
observation, we can write 

x(t) -x = Pit- s)Px - Mxq = (P(t - s) - M)x(s), 

where the last equality follows from the fact that 

PM = MP = M, P is doubly stochastic. (7) 

Let u B be a strategy identical to u A except at link where ufj = and u B = 1. Let the 

matrix B be the system matrix corresponding to u B , and define the doubly stochastic matrix 
Q(t) := e Bt , t > 0. It follows that: 

A i:j > Bij = 0, A kl = B kl V£3{k,l)^{i,j). (8) 

We want to show that switching to strategy u B at some time t* E [s,s + As], can improve the 
utility of the adversary. Formally, we want to prove the following inequality: 

s+As 

k(t)\(P(t-s)-M)x(s)\ 2 dt 



< I k(t)\(P(t- s)- M)x(s)\ 2 dt 

s+As 

+ / k(t) \(Q(t- t*) - M)P(t* - s)xis)\ 2 dt, 



or equivalently 



AS kit) ■ [\(Q(t - t*) - M)Pit* - s)x(s)f 



t* 



- | (P{t - s) - M)x(s) | 2 ] dt > 0. (9) 
Using (|7]) and the semi-group property, (|9]) simplifies to 

kit) ■ x(s) T A(t, t*)x(s)dt > 0, (10) 



where A(t, t*) = Pit* - s)Q(2(t - t*))P(t* -a)- P(2(t — s)). A sufficient condition for ([T0]> 
to hold is 

hit, x(s)) = x(s) T A(t, t*)xis) > 0, for t > t\ (11) 
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As t 4 0, we can write P(t) = I + tA + O (t 2 ), where O (t 2 ) /t < K for sufficiently small t and 
some finite constant K. We therefore have 

A(t, t*) = (I+ (t* -s)A + (t 2 )) (I + 2{t - t*)B 
+0 (t 2 )) (I + (f -s)A + (t 2 )) - (/ + 2(t - s)4 
+£> (t 2 )) = 2{t - + 2{t - s)A - 2{t - s)A 
+0 (t 2 ) = 2{t - t*){B - A) + O (t 2 ) . 

For sufficiently small t and t*, the first term dominates the second term. For any symmetric L with 
LI = 0, the quadratic form exhibits the following form: x T Lx = — YH=\ Tlik=i Lm{ x i — Xk) 2 , 
for any x G W 1 . Using ([8]), we can then write 

n l-l 

h(t,x(s)) = 2(t-f)J2J2^ Akl ~ Bkl ^ Xl ^- Xk ^ 2 

1=1 k=l 

= 2(t - t*)Aij ( Xj (s) -Xi{s)f. (12) 

Hence, if there is a link such that Xi(s) ^ Xj(s), there exists t*, t such that h(t,x(s)) > 
for t G (t*, t\ . By the semi-group property, we can write 



Thus, for any t > 0, not necessarily small, and by selecting r to be sufficiently large, we have: 
P[tX = (i + U + O (tX\ r ~ I + tA + O (fl 



By following the same analysis as above with this approximation, we conclude that we can 
always find t* > s such that h(t,x(s)) > for t > t*. Since s was arbitrary, we conclude that 
the optimal strategy must satisfy N u * (£) = i for all t, given that each of the £ links connects two 
nodes having different values. If no such link exists at a given time s, the adversary does not 
need to break additional links, although breaking more links does not affect optimality because 
h(t, x(s)) — in such case. There are two cases under which the adversary cannot find a link to 
make h(t, x(s)) > 0: (i) The graph at time s is one connected component. In this case, the nodes 
have already reached consensus and N u * = < i. This is a losing strategy for the adversary 
as it failed in preventing nodes from reaching agreement; (ii) The graph at time s has multiple 
connected components, and the number of links connecting the components is less than t. The 



March 13, 2013 



DRAFT 



8 



adversary here possesses a winning strategy with N u * < I, as it can disconnect Q into multiple 
components and prevent consensus. 

Thus far, we have shown that the adversary can improve his utility if he switches at some 
time t* G [s, s + As] from strategy A to strategy B (where strategy B corresponds to the 
proposed optimal control). Now, we want to show that switching to strategy B guarantees an 
improved utility for the adversary regardless of how the original trajectory A changes beyond 
time s + As. To show this, we will assume that from time s + As onward, strategy B will mimic 
the original strategy. Assume that strategy A switches to another strategy C (hence, strategy B 
will also switch to strategy C). Let us denote the system matrix corresponding to strategy C by 
R(t) := e ct . Let us also restrict our attention to a small interval [s + As, s + 2As] over which 
we can assume that the system is time-invariant. 

We want to prove the following inequality: 



As before, it suffices to prove that the integrant (in particular, h — I 2 ) is positive. Let us now 
expand both l x and I 2 . 



h = x(s) T P(t* - s)Q(s + As - t*)(R(t - (s + As)) - M)(R(t - (s + As)) - M) 
Q(s + As - t*)P(t* - s)x(s) 
= x(s) T P(t* - s)Q(s + As - t*)(R(2(t - (s + As))) - M)Q(s + As - t*)P(t* - s)x(s) 
= x(s) T (P(t* - s)Q(s + As - t*)R(2(t - (s + As)))Q(s + As - t*)P(t* - s) - M)x(s). 




\(R(t - (s + As)) - M)P{s + As - s)x(s)| 2 dt > 0. 
■> ' 



Similarly, 



I 2 = x(s) T (P(As) J R(2(t - (s + As)))P(As) - M)x(s) 
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Further, we have 

l x -I 2 = x(s) T (P(t* - s)Q(s + As - t*)R(2(t - (s + As)))Q(s + As - t*)P(f - s) 

" v ' 

Fi 

- P(As)R(2(t - (s + As)))P(As) )x(s). 

^2 

Before we perform a first-order Taylor expansion to the above terms, let us define the following 
quantities: 

n = t*-s, r 2 = (s + As)-t\ r 3 = t- (s + As), 

where t* G [s,s + As] and t G [s + As, s + 2 As]. 

Recall that we write f(x) — O (g(x)) as x — > a if 3 constants M, 5 such that 

|/(x)| < M\g(x)\, for all x satisfying \x — a\ < 5. 

Also, recall that following properties: 

. f(x)0(g(x)) = 0(f(x)g(x)) 

• c • O (/(#)) = (f(x)), c is a constant 
Using the above definition, we can prove the following claims. 

Claim 1: As As — > 0, we have: 

1- If f{r u As) = O (7f ), then f(r t , As) = O (As 2 ) , i G {1, 2, 3} 

2- If /(r,,r„ As) =r t O(rf), then /fa, 7*, As) = O (As 3 ) , i,j G {1,2,3} 
Proof: We proceed by using the above definition and properties. 

1- We have that /fa, As) < Mr 2 < MAs 2 . Hence, fin, As) = O (As s ). 

2- /fa, t,, As) = r.O (r 2 ) = (r.r 2 ) . Hence, /fa, r,-, As) < Mr,r 2 < MAs 3 and /fa, Tj , As) = 
0(As 3 ). 
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We can now expand Fx and F 2 as follows. Note that, As ->■ 0, / + O (As 2 ) + O (As 3 ) = 
f + 0(As 2 ). 

F x = (l + Tl A + (r 2 ) )(l + r 2 B + (r 2 )) (/ + 2r 3 C + O (r 2 ) ) (I + r 2 B + O (r 2 ) ) 
(/ + t 1 A + C(t 1 2 )) 

= (/ + t x A + t 2 B + (As 2 )) (/ + 2r 3 C + O (As 2 )) (/ + t x A + t 2 B + O (As 2 )) 
= (/ + nA + r 2 B + 2r 3 C + O (As 2 )) (/ + + t 2 B + O (As 2 )) 
= 1 + 2nA + 2t 2 B + 2r 3 C + O (As 2 ) . 



F 2 = (I + AsA + O (As 2 )) (I + 2r 3 C + O (t*)) (I + AsA + O (As 2 )) 
= (I + AsA + 2t 3 C + O (As 2 )) (I + AsA + O (As 2 )) 
= I + 2AsA + 2t 3 C + O (As 2 ) . 

Hence, we have 

Fx-F 2 = 2 (tx - As) A + 2r 2 B + O (As 2 ) , 

= 2r 2 (B - A) + O (As 2 ) 

= 2 ((s + As) - £*) (B - A) + O (As 2 ) 

and therefore, using (8), we obtain 

Ix - h = 2 (s + As - t*) Aij (xj(s) - Xi(s)) 2 > 0, 

as required. 

It remains to show that the links the adversary breaks have the highest i%-(i) values. Let us 
again restrict our attention to the interval [s,s + As] where the adversary applies strategy u A . 
Assume (to the contrary) that the links the adversary breaks over this interval are not the ones 
with the highest w%j{t) values. In particular, assume that the adversary chooses to break link 
(k,l), while there is a link such that Wy(t) > Wki{t), t G [s, s + As]. Assume that the 
adversary switches at time t* £ [s, s + As] to strategy u B by breaking link (i,j) and unbreaking 



link (k,l). Then, (12) becomes h(t,x(s)) = 2{t — t*) (wij(s) — wik(s)). Hence, by following the 
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same arguments as above, we conclude that breaking (k, I) is not optimal. The proof is thus 
complete. ■ 

Remark 1: Potential theory aids in providing an interpretation of the result of Thm[TJ Con- 
sider an electrical network with Xi being the voltage at node i with respect to a fixed ground 
reference. Then, (xj — xA represents the potential difference (or voltage) V across link 
The edge weight represents the conductance of the link. It follows that the weight 10^ is in 
fact the power P dissipated across link Hence, the adversary will break the links with the 
highest power dissipation. 

Theorem 2: The optimal strategy derived in Thm [TJ satisfies the canonical equations of the 
MP. 

Proof: Recall that the MP requires us to find the lowest fa's whereas Theorem [T] dictates 
that we find the largest Wi/s. Let us define the terms Wy = —Wij. Thus, it is sufficient to show 
that Wij < Wki implies that fa < fa. To do so, we need to perform a first-order Taylor expansion 
for x(t) and p(t) over the interval [s = T — As, T), with At > small. Over this interval, the 
system is time-invariant. Let A denote the system matrix over this interval. Then, from ([4]) and 
([4]), we can write 

x(t) = e A{t - s) x{s) (13) 

r T 

p(t) = 2/ e- A{t - T \x(T) -x)dr. (14) 



Let P{t - s) := e A( *~ s) =/ + (*- s)A + O (As 2 ). We can then simplify re-write the above 
expressions as 

x(t) = P(t-s)x(s) 

= [I + {t - s)A]x(s) + O (As 2 ) , 



and 



pit) = 2 J P(r-t) [P(t -s)- M]xis)dr 

= 2 J [P(2t - t - s) - M)x{s)dr 

= 2 J [I + i2r-t-s)A-M]xis)dr + 0(As 2 ) 

pit) = [2(T-t)/ + 2(T-t)(T-s)A-2(T-t)M]x(s) + C(As 2 ). 
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Define s) := t — s and write 

x(t) = [I + £(t,s)A}x(s) + 0(As 2 ) (15) 
p(t) = [2i(T,t)I + 2i(T,t)i(T,s)A-2i(T,t)M}x{s) + 0{As 2 ). (16) 

Further, define the matrices 

G :— I + £(t, s)A, H := 2£{T, t)I + 2£(T, t)£{T, s)A - 2£(T, t)M. 

Ignoring the higher order terms O (As 2 ) for simplicity, we can write 

= a ij x(s) T (g i - gj )( gj - gj) T x(s) 

fij = a i3 x(s) T (hi - h 3 )(g 3 - g 3 ) T x(s), 

where gf , hf are the i-th row of G and H, respectively. Using the definitions of G and H, we 
obtain 

(9 i - 9j)(9j - 9j) T = -{h ~ Wi ~ - s)[(Ii - - A 3 ) T + (Ai - A 3 )(I t - Ij) T ] 

—£(t, s) 2 (Ai - Aj)(Ai - Aj) T . 
The last term is quadratic so we can lump it into O (As 2 ). We then have 

"ij(9i - 9j)(9j - 9jf ~ aki{9k - gi)(gi - g k f = {a k i{h - h){h - h) T - aijih - I 3 )(Ii - I 3 ) T ) 
+(a kl (I k - I{){A k - Ai) T - a^ih - 7,)(A - A 3 ) T )t(t, s) 
+(a kl (A k - A t )(I k - h) T - a tJ (A t - A 3 )(I t - Ij) T )^t, s) + O (As 2 ) . 
Similarly, we have 

aij(hi - hj)(gj - gj ) T - a kl (h k - hi)(gi - g k ) T = (a kl (I k - Ii)(I k - h) T 

-a t3 {I t - I 3 )(h - I 3 ) T )2^T,t) + (a kl (I k - I t )(A k - AA T - a l3 {h t - I 3 )(A t - A 3 ) T )2^T,t)^t, s) 

+(a kl (A k - - h f - a t3 (A t - A 3 )(I t - J,) T )2£(T, t)£(T, s) + O (As 2 ) . 

Let r\ = a k i(I k - Ii)(I k - Iif - a i3 (Ii - I 3 )(Ii - I 3 ) T , T 2 = a M (I k - h)(A k - A t ) T - a i3 (Ii - 
Ij)(Ai - A 3 ) T , and T 3 = a kl {A k - Ai)(I k - I t ) T - a i3 {Ai - A 3 ){Ii - I 3 ) T . We now have 

Wij-wu = x(s) T (T 1 + ^t,s)T 2 + ^t,s)T 2 )x(s) + 0(As 2 ) 

fij ~ hi = x(s) T (2e(T, i)rx + 2f (T, t)£(t, s)r 2 + 2£(T, t)£{T, s)T 3 )x(s) + O (As 2 ) . 
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But £(T,t)£(t,s) and £(T,t)£(T, s) are of order As 2 so we can also lump them into O (As 2 ) 
to obtain 

h - fa = xisfmT, t)T l )x(s) + O (As 2 ) . (17) 
If Wij — Wki < 0, and since £(T, t) < 0, we can write 

2£(T,t)(^-^) = x(s) T (2aT 1 t)T l +2aT } t)at,s)T 2 +2aT,t)at,s)T 2 )x(s)+0 (As 2 ) < 0, 
or 

x(s) T (2^(T,t)T 1 )x(s) + O (As 2 ) < 0, 

but the left hand side is — fa; hence, Wij < wu =^ fij < fa as required. 

So far, we have verified the claim over the interval [s, T] only. We need to verify that the 
claim holds over the interval [r = T — 2 As, s). If the claim holds over this interval, then it can 
be generalized for the entire horizon of the problem [0, T\. The only complication that arises 
when studying this interval is that the terminal condition, i.e. p(s), is not forced to be zero as 
in [s,T]. Let the system matrix over [r, s] be B. Then, the state and costate are give by 

x (t) = e B{t - r) x{r) 

p(t) = e- B{t - r) p(r) -2 J\- b ^- t \x(t) - x)dr. 

Solving for p(r) interns of p(s) and substituting back, we can write p(t) interns of p(s) as 
follows: 

p(t) = e- B{t ~ s) p(s) + 2^ e- B{t ~ T \x(T) - x)dr. 



The integral term on the above expression is similar to that in ( 14), and the same analysis above 



applies to it. We can obtain an expression for p(s) from (14) to arrive at the following solution 
over [r, s] : 

p(t) = 2 J e~ B{ t- T \x B (T) - x)dr +2 J e- B ^-^- A( - s - T \x A (T) - x)dr 

^ v ' ^ v ' 

h h 
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Let Q(t) = e Bt = I + tB + (As 2 ). Then 




((s - t)I + (s-t)(s- r)B - (s - t)M) x(r) + O (As 2 ) 
/ Q(s- t)P(r - s) [P(t - r) - M]x A (s)dr, 



[Q(s - t)P(2r - s - r) - M}x(s)dr 



[(I + (s - t)B){I + (2r - s - r)A) - M]x j4 (s)rfr + O (As 2 ) 



[/ + (2r - s - r)A + (s - i)£ - M]x A (s)rfr + O (As 2 ) 



= ((T - s)I + (T 2 - s 2 - (s + r)(T - s)) A + (s — t)(T — s)B — (T — s)M) x A (s) + C (As 2 ) . 



where we have used the fact T — s = s — r = As. Since the state is continuous, we have that 

xa(s) = xb(s) or xa(s) = e B ^ s ~ r ^x(r). We can then write 



I 2 = ((T-s)I+(T 2 -s 2 -(s + r)(T-s))A + (s-t)(T-s)B-(T-s)M) 
•(/ + (s - r)B)x(r) + O (As 2 ) 
= ((T - s)I + (T 2 -s 2 -(s + r)(T- s)) A + (T - s)(2s - t - r)B — (T — s)M) 
•x(r) + O (As 2 ) 



p(f) = (2(T - t)I + 2(T - s)(T - r)A + 2(T - s)(3s - 2t - r)B - 2(T - t)M) x(r) + O (As 2 ) 
= (2£(T, f ) J + 2£(T, s)£(T, r) A + 2£(T, s) (2£(s, f ) + £(s, r))S - 2£(T, t)M) x(r) + O (As 2 ) . 



Following similar steps to the above, we can derive the following expressions over the interval 



Wij-Wki = x(r) T (r 1 + ^t,r)r 2 + ^(t,s)r 2 )x(r) + 0(As 2 ) 
fij - hi = x(sf(2£(T, 0r x + 2£(T, t)^(t, r)r 2 + 2£(T, s)(2£(s, t) + £(s, r))r 3 
+2£(T,s)(T,r)r 4 )a;(r) + O (As 2 ) , 

where r\ = a H (4 - h){h - h) T - %(^ - - T 2 = a kl (I k - h){B k - B t ) T - 
aij {Ii - IJiBi - B 3 ) T , T 3 = a kl (B k - B t )(I k - h) T - aij (Bi - £?,)(/, - J,) T , and T 4 = 



Summing both integrals, we obtain 
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a M (A k - Ai)(I k - I t ) T - aij(Ai - Aj)(Ii - Ij) T . Note again that f tj - f H can be simplified to 

fij - fki = x(r) T (2aT,t)T l )x(r) + O (As 2 ) . (18) 

Thus, by the same argument used over [s,T], we conclude that — Wki < implies that 
fij — fki < 0. From the structure of p(t), and the above analysis, we conclude that — f^i will 
always have the same form as that in ( fl"8"j ). Hence, we conclude that the claim holds over the 
entire horizon [0, T]. ■ 
We now provide a geometric property of u*(t). 

Lemma 1: (Scale Invariance) If u*(t) is the optimal solution to ([I]) with x(0) = x , then it 
is also optimal when starting from x(0) = c • Xq, c G M. 

Proof: Let x(t) and p(t) be state and co- state vectors corresponding to the initial state 
x = c ■ x . Then, by (|4]), ([5]), and uniqueness of the transition matrix, we have: 

x(t) = $ x (t,0)x = c-x(t), 

p(t) = c ' 2 ^" %(t,r)k(T)[^ x (r,0)x o — xdr] = c- p(t). 

Hence, /y = Oij(pj(t) - pi{t))(xi(t) - Xj(t)) = c 2 ■ aij(pj(t) - Pi{t))(xi(t) - Xj(t)). Hence, 
s g n (A?) = sgn^J, Vz, j. U 
Consider the following sets for i G {1, Yl]=o (") ) : 5j = {i 6 1™ : «j = arg max ueW J(u), x(0) 
The set Si corresponds to the set of initial conditions starting from which the solution to ([T|) is 
stationary. In view of Lemma [T] we conclude that the sets Si are linear cones. 

III. Attack II: An Adversary Capable Of Corrupting Measurements 

Assume now that the adversary is capable of adding a noise signal to all the nodes in the 
network in order to slow down convergence. The dynamics in this case are: 

x(t) = Ax(t) + u(t), x(0) = x . (19) 

We assume that the instantaneous power u(t) T u(t) =: \u(t)\ 2 that the adversary can expend 
cannot exceed a fixed value P ma x- We also assume that the adversary has sufficient energy E max 
to allow it to operate at maximum instantaneous power. Accordingly, the strategy space of the 
adversary is U = {u(t) G C x [0, T) : \u(t)\ 2 < P max } , where C x [0, T] is a Banach space when 
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endowed with the following norm: \\x\\ cl = \\x\\ L + ||x|| z . Thus, the adversary's problem is 

max u(t)eW J(u) (20) 

n 

s.t. x(t) = Ax(t) +u(t), Au = - A iJ' 

Aij = Aji, Aij > 0, A i:j > (i, j) e S. 
The Hamiltonian in this case is given by 

H(x,p,u) = k(t) \x(t) - x\ 2 + p(t) T (Ax(t) + u(t)) 

+\(t) {\u(t)\ 2 - P max ) } 

where \(t) is a continuously differentiable Lagrange multiplier associated with the power con- 
straint. As before, we let x(t),p(t) e C^T]. Here, \(t) must satisfy 

A(t)<0, \(t) {\u(t)\ 2 - P max ) =0. 

The first-order necessary conditions for optimality are: 

d 

P(t) = —Q-H(x,u,p,t) 

= -2k(t)(x(t) -x)- Ap{t), p{T) = 
x(t) = Ax{t) + u{t), x(0) = Xq 

^-H(x,u,p,t) = 2X(t)u(t) +p(t) = 0. (21) 
To find u*(t), consider the following cases: 



Case 1: A(t) < ==► \u{t)\ 2 = P max . Using (gTp, we obtain A(t)|w(t)| 2 = -\u(t) T p{t); 
hence, 

A(t) = -^^«(t) T p(t), (22) 

which we can then use to solve for the optimal control: 

p(t) E max p(t) 



U*(t) = Prr 



'u{t) T p{t) t u(t) T p(ty 

Remark 2: The optimal strategy u*(t) is the vector of maximum power that it is aligned with 



pit). To see this, note that (22) implies that u{t) p(t) > 0, because X(t) < 0. Hence, the vectors 



u*(t) and p{t) are aligned. Define the unit vector p(t) = pit)/ \p\. Then, we can further write 

U it) = : — ; p = \JPmax ■ P. (23) 

\u\ 
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Hence, the adversary's optimal solution in this case is to operate at the maximum power available. 
Case 2: \u(t)\ 2 < P max A(i) = 0. Using (21), we obtain p(t) = 0. In this case the 

control is singular, since it does not appear in J^H = 0. But since p(t) = for all t, all its time 
derivatives must also be zero: 

" ''" p(t) = -2k(t) (x(t) -x)- A T p(t) = 0, 



dt du 

x (t) - x = 0. (24) 
The conditions obtained by taking the time derivatives are also necessary conditions that must 



be satisfied at the optimal trajectory. However, (24) violates the initial condition. In order to 

resolve this inconsistency, we set the control at t = to be an impulse, Ui(t) = c- 5(t), in order 

to make x(0) = x, where c e R is chosen to guarantee |w(t)| 2 < P ma x- Note that we still have 

not recovered the control, and therefore we need to differentiate again 

d? dH ... , . . . . 
__ = l(t) = ^ W+B ( t ) = 0, 

u (t) = -Ax(t) = -Ax = 0. (25) 



Remark 3: Note that x(t) = x leads to having u{t) = 0. This result matches intuition; when 
the nodes reach consensus, J(u) = for all u(t) £ hi. Hence, no matter what the control is, the 
utility of the adversary will always be zero. Thus, expending power becomes sub-optimal, and 
the optimal strategy is to do nothing. 

Because the adversary attempts to increase the Euclidean distance between x(t) and x, we can 
readily see that u(t) = cannot be optimal, unless x(t) = x. The following lemma proves this 
formally. 

Lemma 2: The solution of (20) satisfies |w(t)| 2 = P m ax- 

Proof: Assume that |wi(t)| 2 < P ma x', then by (24) and (25), J(u\) = 0. Consider another 



solution which satisfies the power constraint with equality. Namely, let U2(t) = J ^^1. Using 
the solution to (19), and by defining the doubly stochastic matrix P(t) = e At , for t > 0, 



x(t) = P(t)x + 



P„ 



-It. 



n 
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In this case, for t > 0, we have 

\x(t) - x\ 2 = xl(P{t) - M) T (P(t) - M)x 



-P max t 2 + 2^^x T (P(t)-M)lt 

-- x T (P(t) 2 - 2MP(t) - M 2 )x + P max t 2 (26) 
= x T P{2t) (/ - M)x + P max t 2 , (27) 



where (26) follows because P(t), t > 0, and M are stochastic matrices, and (27) follows from 
^ and the semi-group property. Being a stochastic matrix, P(2t) is positive semidefinite (psd). 
Also, I — M is a Laplacian matrix; therefore, it is also psd. Further, note that 

P(2t)(J - M) = P{2t) - MP(2t) = (I - M)P(2t). 

Hence, P(2t)(I - M) is also psd, and therefore x%P(2t)(I - M)x > for t > 0. This in turn 
implies 

J(u 2 ) = [ k(t)[x^P(2t)(I-M)x + P max t 2 ]dt 
Jo 

> ^T 3 >J( Ul ) = 0. 
3 

We conclude that not utilizing the power budget available yields a lower utility for the adversary. ■ 
With Lemma [2] at hand, it remains to determine the co-state vector in order to completely 
characterize u*(t). To do so, we will invoke Banach's fixed-point theorem. To this end, we will 
work with the scaled utility J(u) = uJ{u), v > 0, without loss of generality. Note that u*(t) in 



(23) is also the solution to the maximization problem of J(u). The co-state trajectory is given 

by rp 

pit) = 2v I k{r)P(T - t)(x(r) - x)dr. (28) 



Substituting (23) and the solution to (19) into p8fc yields 



p(t) = g{t) + 2v^P^ x [ [ k(r)P(2r - (t + s))p{s)dsdr, 

Jt Jo 



where g{t) = 2v J? P(t - t)/c(r)(P(r)x - x)dr. Note that 2r - (t + s) > for < s < r, 
t < r < T, and hence P(.) is a well-defined doubly stochastic matrix over the region of 
integration. We define the mapping T(p)(t) := p(t). By its structure, it is readily seen that 
T{p)(t) : C^OjT] — > C^fO, T]. The following lemma aids in obtaining the co-state vector. 
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Lemma 3: Let T(x)(t) : = k(t) f* P(s)x(s)ds, where P(t) is a doubly stochastic matrix, and 
fix x(t) E C^T]. Then 



T(x) < sup tk(t) ■ \\x\\ L . 

Loo 0<t<T 



Proof: We have: 



t{x) 


= sup 


k{t) [ P(s)x(s)ds 




Loc 0<t<T 


Jo 



= sup k(t) sup / y j Pjj(s)xj(s)ds 

0<t<T l<i<n Jo ~[ 

pt n 

< sup k{t) sup / y^ Pjjjs) \xj(s) \ ds 

0<t<T l<i<nJo ~[ 

< sup k(t) sup / y^Pij(s) ] sup \xj(s)\ds 

0<t<T l<i<nJo J !<i<™ 

= sup k(t) / sup \xj(s)\ds 

0<t<T Jo l<j<n 

< sup k(t) / sup sup \xj(s)\ds 

0<t<T Jo 0<s<Tl<j<n 

= sup tk{t) ■ \ \x\\ L , 

0<t<T 

where (a) follows from Holder's inequality. ■ 

Theorem 3: By choosing v < 1 - where k = sup 0<t<T tk(t) and k = sup 0<t<T jj Tk(r)dT, 
the mapping T(p)(t) : C^,T\ ->■ C l [Q,T\ has a unique fixed point p*(t) E C l [Q,T] that can 
be obtained by any sequence generated by the iteration Pk+i(t) = T(pk)(t), starting from an 
arbitrary vector p (t) G C^T]. 

Proof: The theorem will follow if for this choice of u, the mapping T is a contraction. 
Consider two vectors y(t),z(t) e C 1 [0,T] and let y(t),z(t) be the corresponding normalized 
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unit norm vectors. Let w = y — z. Then 

' \\T{y)-T{z) 



lei 



sup k{t) sup 

0<t<T Ki<n 



+ sup sup 

0<t<T Ki<n 



/ y^^i(*- s )%( s )^ 

Jo j=1 

/ k ( T ) / V^(2r- (t + s))%(s)dsdr 
A io J=1 

< sup tk(t) \ \w\\ L + sup sup / k{r) 

0<t<T °° 0<t<T l<i<n J t 

n 

^2 p ij( 2T ~ (t + s)) \wj(s)\ dsdr, 



where the last inequality follows from Lemma [3] Using arguments similar to those used in 
proving Lemma [3j we have: 

1 



\\T(y)-T(z) 



< I sup tk(t) + sup / Tk(r)dT ) \\w\u 

\0<t<T 0<t<TJt 



(29) 



<(k + k) \\y-z\\ Loo < 2zy V P m ax{k + k) || 

where the second inequality follows from the properties of similar triangles. We readily see that 
by selecting v < , 1 - the last inequality implies that T(p)(t) is a contraction mapping. 
Because C^O,! 1 ] endowed with ||.|| c i is a Banach space, Banach's contraction principle guar- 
antees the existence of a unique fixed point p*(t) E C 1 [0,T] which can be obtained from the 
iteration Pk+i(t) = T(pk){t) as k — > oo, for any initial point. ■ 

IV. Numerical Results 

In this section, we provide a numerical example for ATTACK-I. We consider the complete 
graph with n = 4. The matrix A(0) is generated at random and is equal to 

/ -2.1293 0.0326 0.5525 1.5442 \ 

0.0326 -1.2191 1.1006 0.0859 

A(0) = 

0.5525 1.1006 -3.1447 1.4916 

\ 1.5442 0.0859 1.4916 -3.1217 / 
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Fig. 1. Effect of ATTACK-I on the convergence to consensus. T — 2, n — 4, I = 2, and xq = [1, 2, 3, 4]. 

We fix £ = 2, T = 2, and x = [1,2,3,4] T - hence, x avg = 2.5. We simulated the network 
using Matlab's Bvp Solver and computed the optimal control using (|6]), which was found 
to be u*(t) = [1, 0, 1, 0, 1, 1] T for t G [0,2]. Indeed, at t = 0, the highest Wij values are 
^13(0) = 2.2101 and u>i4(0) = 13.8979 which confirms the conclusion of Thm [T] In this 
particular example, if 13, Wi^ remain dominant throughout the problem's horizon, and hence the 
control is stationary. Fig. [T] simulates the network at hand with and without the presence of the 
adversary. Note that the adversary was successful in delaying convergence. Since both links the 
adversary broke emanate from node 1, xi(t) is far from consensus. 

V. Conclusion 

We have considered two types of adversarial attacks on a network of agents performing 
consensus averaging. Both attacks have the common objective of slowing down the conver- 
gence of the nodes to the global average. Attack-I involves an adversary that is capable 
of compromising links, with a constraint on the number of links it can break. Despite the 
interdependence of the state, co- state, and control, we were able to find the optimal strategy. We 
also presented a potential-theoretic interpretation of the solution. In ATTACK-II, a finite power 
adversary attempts to corrupt the values of the nodes by injecting a signal of bounded power. 
We assumed that the adversary has sufficient energy E max to operate at maximum instantaneous 
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power and derived the corresponding optimal strategy. It would be interesting to consider the 
case when E max < T ■ P ma x, when the adversary cannot expend P ma x at each time instant. This 
will be explored in future work. 

References 

[1] J. N. Tsitsiklis, "Problems in decentralized decision making and computation," Ph.D. Dissertation, M.I.T., 1984. 

[2] V. D. Blondel, J. M. Hendrickx, A. Olshevsky, and J. N. Tsitsiklis, "Convergence in multiagent coordination, consensus, 

and flocking," in Proc. Joint 44th IEEE Conf. Decision and Control and European Control Conf., December 2005. 
[3] R. Olfati-Saber, "Flocking for multi-agent dynamic systems: Algorithms and theory," IEEE Trans. Automat. Contr., vol. 51, 

no. 3, pp. 40-420, 2006. 

[4] A. Jadbabaie, J. Lin, and A. S. Morse, "Coordination of groups of mobile autonomous agents using nearest neighbor 

rules," IEEE Trans. Automat. Contr., vol. 48, no. 6, pp. 988-1001, 2003. 
[5] S. Li and T. Ba§ar, "Asymptotic agreement and convergence of asynchronous stochastic algorithms," IEEE Trans. Automat. 

Contr, vol. 32, no. 7, pp. 612-618, 1987. 
[6] A. Nedic, A. Ozdaglar, and A. Parrilo, "Constrained consensus and optimization in multi-agent networks," IEEE Trans. 

Automat. Contr., vol. 55, no. 4, pp. 922-938, 2010. 
[7] M. O. Jackson and B. Golub, "Naive learning in social networks: Convergence, influence and wisdom of crowds," American 

Economic J.: Microeconomics, vol. 2, no. 1, pp. 112-149, 2010. 
[8] A. Kashyap, T. Ba§ar, and R. Srikant, "Quantized consensus," Automatica, vol. 43, no. 7, pp. 1192-1203, 2007. 
[9] L. Xiao, S. Boyd, and S.-J. Kim, "Distributed average consensus with least-mean-square deviation," J. Parallel and 

Distributed Computing, vol. 67, no. 1, pp. 33-46, 2007. 
[10] B. Touri and A. Nedic, "Distributed consensus over network with noisy links," in Proc. 12th Internat. Information Fusion 

Conf., 2009, pp. 146-154. 

[11] A. Sarwate and A. Dimakis, "The impact of mobility on gossip algorithms," IEEE Trans. Inform. Theory, vol. 58, no. 3, 
pp. 1731-1742, March 2012. 

[12] R. Olfati-Saber and R. M. Murray, "Consensus problems in networks of agents with switching topology and time-delays," 

IEEE Trans. Automat. Contr, vol. 49, no. 9, pp. 1520-1533, 2004. 
[13] A. Nedic and A. Ozdaglar, "Convergence rate for consensus with delays," J. Global Optimization, vol. 47, no. 3, pp. 

437-456, 2010. 

[14] B. Touri and A. Nedic, "On ergodicity, infinite flow, and consensus in random models," IEEE Trans. Automat. Contr, 

vol. 56, no. 7, pp. 1593-1605, July 2011. 
[15] , "Product of random stochastic matrices," arXiv:1009.3522, 2011. 

[16] , "When infinite flow is sufficient for ergodicity," in Proc. 49th IEEE Conf. Decision and Control, December 2010, 

pp. 7479-7486. 

[17] S. Sundaram and C. Hadjicostis, "Distributed function calculation via linear iterative strategies in the presence of malicious 

agents," IEEE Trans. Automat. Contr, vol. 56, no. 7, pp. 1495-1508, July 2011. 
[18] F Pasqualetti, A. Bicchi, and F. Bullo, "Consensus computation in unreliable networks: A system theoretic approach," 

IEEE Trans. Automat. Contr, vol. 57, no. 1, pp. 90-104, January 2012. 



March 13, 2013 



DRAFT 



23 



[19] A. Teixeira, H. Sandberg, and K. Johansson, "Networked control systems under cyber attacks with applications to power 

networks," in Proc. American Control Conference, July 2010, pp. 3690-3696. 
[20] J. Norris, Markov Chains. New York.: Cambridge Series in Statistical and Probabilistic Mathematics, 1997. 



March 13, 2013 



DRAFT 



